DOCUMENT RESUME 



ED 237 513 



TM 820 753 



AUTHOR 
TITLE 

INSTITUTION 
REPORT NO 
PUB DATE 
NOTE \ 
AVAILABLE FROM 

PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



! 



Wilson , Kenneth M, 

GMAT and GRE Aptitude Test Performance 
Primary Language and Scores on TOEFL* 
Educational Testing Service , Princeton , 
ETS-RR-B2-28; TOEFL-RR-12 
Oct 82 

78p. i < 

Off ice 9 Educational Testing 
08541, 



in Relation to 



N,J, 



TOEFL Program 
Princeton , NJ 



Service , 



Reports , - Researeh/Teshnical '(143) 



MF01/PC04 Plus Postage, 

Admission Criteria; ^College Entrance Examinations; 
^English (Second Language) ; *Foreign Students; 
^Graduate Study; Language frosts ; Post secondary 
Education ; , Scores ; Testing Programs; Test Use \ 
Graduate Management Admission Test; Graduate Record 
Examinations; *Native .Language; *Test of English as a 
Foreign Language - * 



ABSTRACT 

This study was designed to describe and analyze (1) 
the performance of foreign candidates taking the Graduate Management 
Admission Test (GMAT) or the Graduate Record Examinations (GRE) 
Aptitude Test in relation to self-reported primary language (English 
vs * other) , and (2) relationships between performance on the 
respective admissions test? and performance on the Test of English as 
a* Foreign Languages (TOEFL) for subgroups of foreign admissions-test 
candidates identified by 'cross-file matching es having also taken 
TOEFL. Data were obtained from files maintained by the three testing 
programs and analyses were based on data for examinees tested during 
the period from September 1977 \through August 1979 * (Author/PN) 
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Kenneth M. Wilson 



The Test of English-as a Foreign Language (TOEFL) was developed in ,1963 by 
a National" Council on the Testing of English as a Foreign Language, which,, 
was formed through the cooperative effort of over thirty organizations, public 
and private, that were concerned with testing the English proficiency of non- 
native speakers of the language applying for admission to institutions in the 
.United States. In 1965, Educational Testing Service (ETS) and the College 
Board assumed joint responsibility for the program and in 1973 a cooperative - 
arrangement for the operation of the program was entered Into by ETS, the 
College Board, and the Graduate Record Examinations (ORE) Board, The 
membership of the College Board Is composed of schools, colleges, school 
systems, and educational associations; QRE Board members are associated' 
with graduate education, • . 

ETS administers the TOEFL program under the general direction of a Policy 
Council that was established by, and is affiliated with, the sponsoring organi- 
zations, Members of the Policy Council represent the College Board and the 
GRE Board and such institutions and agencies as graduate schools of 
business, junior and community colleges, nonprofit educational exchange 
agencies, and agencies of the United States government, 



A continuing program of research related to TOEFL is carried out under the 
direction of the TOIFL Research Committee, Ms six members include repre- 
sentatives of the Policy Council, the TOEFL Committee of Examiners, and 
distinguished Engfish-as-a-second language specialists from the academic 
community, Currently the committee meets twice yearly to review and ap- 
prove proposals for test-related research and. to set guidelines for the entire 
scope of the TOEFL research program. Members of the Research Committee 
serve three year terms at the Invitation of the Policy CounciU the chair of the 
committee serves on the Policy Council, s 

Because the studies are specific to the test and the testing program, most of 
the actual research is conducted by ETS staff rather than by outside re- 
searchers. However, many projects require the cooperation of other institu* 
lions,, particularly those with programs in; the teaching of English as a foreign 
or second language* Representatives of such programs who are interested in 
participating in or conducting TOEFL-related research are invited to contact 
the TOEFL program office, Local research may sometimes' require pecess to 
TOEFL data. In such cases, the program may provide this data following 
approval by the Research Committee^ All TOEFL research projects must 
undergo appropriate ETS review to ascertain that ttle confidentiality of data 
will be protected. - 

Current (1981-82) members of the TOEFL Research Committee Include the fol- 
lowing; " 

G. Richard Tucker (chair) Center for'Applied Linguistics 
Louis A. Arena University of Delaware 

H, Douglas Brown Univeuity of Nlmoisat Urbana-Champagne 
Frances B, Hinefotis ' University of California at Los Angeles 
Diane Larsen-Freeman The Experiment in International Living 
David S, Sparks University of Maryland 
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The Study in Brief 



This study was "designed to describe and analyze, (a) the performance 
of foreign candidates taking the Graduate Management Admission Test (GMAT) 
or the Graduate Record Examinations (GRE) Aptitude Test in. relation to 
self-reported primary language (English vs* other) , and ,(b) relationships 
between performance on the ^respective admissions tests and performance on 
the Test of English as a Foreign; Language (TOEFL) for subgroups of foreign 
admissions-test candidates Identified by cross-file matching as having 
also taken TOEFL (which is recommended as an aid in assessing the level of 
proficiency in English, of individuals whose English language facility is 
questionable) • 

Data for the -study were obtained from files maintained by the three 
testing programs and analyses^ were based on data for examinees tested 
during the period from September 1977 through August 1979, The foreign 
candidates were classified according to self-reported information as 
being either candidates for whom English was the primary language (EFL 
candidates) or candidates for whom English was the second language 
(ESL candidates), EPL candidates were those who reported that they 
communicated better in English than in any other language (GRE background 
question) or that the language in which they were most fluent was .English 
(GMAT background question); ESL candidates were those who reported better 
communication or greatest fluency in a language other than English * 

Generally * parallel analyses were conducted for each admissions- test 
sample* First , attention, was focused on differences in the levels and 
patterns of admissions test scores for foreign EPL and foreign ESL 
candidates—subgroups assumed to differ in average level of "developed 
proficiency in English," Then* within each admissions test foreign- 
candidate sample* detailed analyses were conducted in subsamples comprised 
of those who also were in the TOEFL candidate pool during the 1977-79 
study period — i,e. , TOEFL /GRE and TOEFL/ GHAT candidates- For the TOEFL/ 
GRE and TOEFL/GMAf subsamples s analyses were conducted to assess the 
extent and nature of the relationships between scores on the respective 
admissions tests (both of which include verbal and quantitative measures) 
and scores on TOEFL; separate analyses were made for EPL and ESL sub- 
groups. Detailed consideration was given to the characteristics of joint 
distributions of scores on trie admissions tests and TOEFL, 



Overview of Findings 

General trends in findings were quite similar across the respective 
admissions- test analyses. Major common themes and related conclusions or 
observations are summarised below* , 

q The performance of foreign candidates on the quantitative 
" '" r sections of GRE and GMAT equaled that of the respective 



general candidate populations; in sharp contrast, -however* 
the verbal admissions—test means of foreign candidates were 
much lower than average. For example, the verbal test score 
means of foreign ESL candidates were in the 14-15 percentile 
range relative to the general GRE/GMAT candidate populations. 

Foreign admissions-test candidates who also take TOEFL are 
highly"" selected members of the TOEFL candidate population 
with respect to TOEFL performance (mean TOEFL total scores of 
553 and 559, respectively, for TOEFL/GMAT and TOEFL/GRE 
candidates) - 

As expected, EFL candidates outperformed ESL candidates on 
TOEFL as well as on the verbal sections of the admissions 
tests. ■ . 

TOEFL total scores, consistent with previous research, were 
found to be 'substantially correlated with GMAT verbal scores 
(rv — .71) ana* GRE verbal scores (r ■ .70) but not with GMAT 
quantitative (r « .39) or GRE quantitative (r = •21) • 

Observed TOEFL/verbal admissions- test correlations Were 
higher for EFL than for ESL subgroups (e.g., TOEFL/GRE verbal 
coefficients were ,74 and .66 and TOEFL/GHAT verbal coeffi- 
cients were .76 and .68 for EFL and ESL subgroups) • Higher 
coefficients for EPL than for the ESL' candidates may be 
attributed , hypotjietlcally , to the likelihood that; because 
of . the greater level of English proficiency in the EPL 
subgroups , the verbal admissions tests are psychometrically 
more appropriate for EFL than for ESL candidates. \ 

The verbal admissions tests were of considerably greater than 
average difficulty for TOEFL candidates,, for whom TOEFL, in 
turn, was of considerably less than average difficulty • 
Measurement efficiency (e.g., reliability) of observed scores 
tends \ to be attenuated in groups scoring at extremes on 
standardised tests generally. Theoretically,; therefore, 
easier "levels** of verbal admissions tests ati d a somewhat 
more difficult level of TOEFL would provide more reliable 
assessments for the TOEMi/admissiens-test populations, hence 
better bases for evaluating observed relationships between 
TOEFL and standard verbal admissions tests. - \ 

Despite the high average level of difficulty of the verbal 
admissions tests' for foreign TOEFL/admissions test candidates^ 
(and for foreign ESL candidates generally), the obtained 
verbal scores have significant psychometric viability and 
verbal test performance increased systematically with 
scores on TOEFL, except in the lowest sector of the joint ' 
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distribution of TQEFL/verbal admissj^ons^^ As 
the level of TOEFL scores and English language proficiency 
increased, TOEFL/ verbal-test relationships became more 
robust^™and the average verbal**test performance of TOEFL/ 
admissions^test candidates began to approximate that of the 
general admissions—test candidate populations • 



General Conclusions T ■■ " 

Certain general conclusions are offered tentatively as working, 
hypo the see for future consideration! 

o Based on findings of the present study and the results of 
scattered ORE and GMAT validity studies involving both U.S. 
citizen and foreign examinees, verbal admissions^test score 
comparisons involving foreign ESL and English-speaking U.S. 
candidates are not; warranted. However, based on the same 
evidence, the observed verbal scores of different foreign 
candidates have a useful degree of interpretability in the 
sense of permitting valid comparisons among them with respect 
to current level of functional ability to perform English 
language verbal reasoning tasks such as those reflected in 
the admissions- test items and tnose involved in carrying out. 



academic work. For foreign 
admissions 1 test scores and 
proficiency in English as a 
maj or factor in accounting 



candidates with very low verbal 
very low TQEFL scores, limited 
second language per se may be a 
for their low verbal scores* 
Given an opportunity to improve their English proficiency, 
their functional ability to carry out the verbal tasks 
required on the admissions tests and in their academic 
assignments may improve. The amount of time required for 
substantive improvement in English language prof iciency 
undoubtedly will vary from candidate to candidate. 

The depressed verbal admissions*~test performance of foreign 
ESL candidates relative to' that of the general candidate 
population may be explained as being due in large part to 
differences between the populations in level of English 
language prof iciencyt— for example, speed of average verbal 
performance of foreign candidates is undoubtedly less than 
that of majority English-speaking candidates. Educational- 
cultural differences .may also be involved, of course. 

i " u ) . A \ ■ 

Research is needed to establish the level of performance of 
foreign candidates on English^language verbal admissions 
tests under essentially un speeded conditions and to extend 
evidence bearing\ on (a) the predictive validity of admissions 
tests for enrolled foreign students and (b) their level of 



academic performance relative to that of majority students 
with comparable admissions^test scores* 

The study findings provide further evidence of the value 
of TOEFL for reliably assessing differences in, English 
proficiency for admissions-test candidates, whose English 
language background is questionable and for^ identifying 
subgroups within a given candidate population- whose verbal 
admissions-* test scores are likely to have differing degrees 
of usefulness in the admission^ process* , * 

There would appear to be a potentially important role for 
separate measures of speed and level of performance on verbal 
tests for foreign ESLjlcandidates. 

Provision d separate reference group score distributions for 
foreign^ and citizen admissions test candidates appears to be 
warrantr * and needed. 



Section 1. Incroduction 



The Graduate Record Examinations (GRE) Aptitude Test and the Graduate 
Management Admission Test (GMAI) aie intended for use in evaluating the 
academic qualifications of applicants for admission to graduate depart^ 
ments and graduate schools of management, respec ti\ ? ely , The GRE Aptitude 
Test* s/hich traditionally has provided measures of developed verbal and 
quantitative reasoning abilities (ORE verbal or GRE— V and ORE quantitative 
or GKE-Q) „ was expanded in 1977 to include a measure of developed analyt- 
leal reasoning ability (GRE analytical or GRE- A ) . The GHAT provides 
measures of verbal and quantitative reasoning (GMAT-V and GMAT^Q) and a 
total score, 

The candidate populations taking these tests ? and che samples used 
in standardization and calibration (scaling), are made up predominantly 
q t individuals who s h <s r ^ u c os*m o n acculturation In t he se ns *s oi ha v i 
been born, reared t and formally schooled in the United States, with 
English as the basic language of discourse. However, these admissions 
testing programs also serve foreign nationals who (a) differ from the 
typical test taker, and among themselves, with respect to cultural and 
educational background and (b) in many i . ^anees, report that they are 
less proficient in English than in some other language* 

o During a recent two— year period (September i977 s August 1979) f for 
example, about 23 percent of GHAT registrants who responded to 
background questions on country of citizenship and language of 
greatest fluency indicated that they were not U*S* citizens; of 
these foreign candidates-, some 48 percent indicated that they were 
"most fluent" in a language other than English (were '"English is 
the secondary iaRgu^ge 1 ' ESL candidates.) while 52 percent were 
"most ' fluent" in English s were "English is the primary language" 
of -EJP.L candidates) * Some 10 percent of all GHAT LU5*. citizen 
candidates wre self ^reported ESL candidates, 



, o Among GRE registrants during the same period, about 13 percent 
were self -repo r ted ly not U.S. citi zens; of these foreign regis-" 
tranti j 5.3 percent reported that they did not J' communicate better 
in English than in any other language" (were ESL candidates j and 
about 47 percent reported the opposite (were self— reported EFL 
candidates); of U-S* citizen candidates almost 3 percent were 
ESL candidates* 

Graduate schools and departments frequently require applicants whose 
English language proficiency is questionable to submit scores on the Test 
of English as a Foreign Language (TOEFL) , which is designed for use as a 
measure of English proficiency for individuals whose "native language" is 
not English. TOEFL yields measures of listening comprehension (ability to 
understand English as it is spoken in the United States), structure and 
written expression (mastery of structural and . grammatical points in 
standard written English) , and reading ^camprghansion vocabulary (word 
knowledge and understanding of a variety of reading" materials) • Scores on 
the three sections are provided along with a total score - " 7 "-- . 




o TOEFL is standardized and calibrated on a population comprised 
predominantly of foreign nationals whose native language is not 
English and who have plans for undergraduate- or graduate-level 
study in the United States or Canada, 

o According to analyses reported in detail elsewhere (Wilson 1982, 
1982a) , about one-half of TOEFL candidates tested in the September 
1977-August 1979 period reported that they did so to facilitate 
plans to enter a graduate-level degree program in the United 
States or Canada* 

The TOEFL program recommends that scores on TOEFL be used as an 
aid in interpreting scores on standard admissions tests presented by 
applicants whose native language is not English and/or in determining 
whether such applicants n* sd remedial work in English* 

This study drew on data -Tom testing files maintained by the GMAT f 
ORE and TOEFL programs at Educational Testing Service (ETS) in oraer to 
describe and analyze the performance of foreign candidates taking the 
two admissions tests in relation to relative proficiency in English as 
reflected (a) by self-reported best or primary language (i*e* , English vs. 
other) and (b) by scores on TOEFL. 

The major objectives of the study were: 

a, to provide evidence regarding the levels and patterns of 
scores on GHAT and the GRE Aptitude Test for foreign candi- 
dates classified by self-reported best or primary language 
(English best [or EPLj vs, English not best lor ESL] ), and 
for citizen ESL candidates — i,e. , those whose self-reported 
"best" language is not English 

b, to analyse the relationship between scores on the two 
admissions tests and scores on TOEFL in samples of candi- 
daf-es identified by cross-file matching as having taken 
both one of the admissions tests and TOEFL* 



Data, Samples f and Study Procedures 

Data for the study were drawn from testing-program files containing 
candidate records for the period from September 1977 through August 1979, 
as follows^ f [ 

o First, the records of candidates who indicated in response to 
background questions that they were not citizens of the United 
States (foreign) or that they were U.S. eitisans but communi- 
cated better In a language other than English (citizen ESL) were 
extracted from the GMAT and GR£ files, (For convenience these 
files are referred to as GHAT foreign or ORE foreign files even 
though they contain records of citizen ESL candidates,) 
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o Second, using name/ sex/birth date identification, records in 
the GHAT and GRE foreign files were matched with records in the 
TOEFL file on candidates for the same two— year period to create 
two subfiles, one for TGEFL/GMAT and the other for TOEFL/ GRE 
candidates—files containing admissions-test and TOEFL records for 
individuals who took both tests during the same two-year period* 

Although the GRE foreign and TQEFL/GRE foreign files included records 
for the entire two-year period, only data for candidates taking the GRE 
Aptitude Test during the 1977-78 testing year were retained for further 
analysis* However, GHAT data for both testing years were included in 
further analyses - 

The records retained for the study were those that included scores 
on the varloufi _testR..Ior aeJLf -reported -nouciti^ens -of the- United States 
and for V * 8 . citizens who reported that English was not their primary 
language, i*e. , citizen ESL candidates - 

Table 1, which provides information regarding sample size, shows that 
a significant proportion of GRE and GHAT candidates who also took TOEFL 
were classified according to their responses to admissions— test background 
questions as EPL candidates* It was not anticipated that the subpopula- 
tion of admissions test candidates taking TOEFL (intended for "individuals 
whose "native language" is not English) would include a significant number 
of members indicating that they were "most fluent" in English (GMAT) or 
that they " communicate [d J better, in English than in any other language" 
(GRE) * However, prospective graduate students whose "native" language 
(e.g., that of parents and the larger society during early years) is 
not English may well be more fluent in English than they are in the 
historically relevant "native language*" It is important to keep in mind, 
of course, that the EPL/ESL distinction connotes relative proficiency— it 
is assumed only that . che^average level of English proficiency for EPL 
candidates is higher than that for ESL candidates* 

o Foreign candidates who report that they are better communicators 
in English than in another language (EPL candidates) are not 
necessarily "natively fluent" in English (though they may be 
expected to exhibit higher scores than their ESL counterparts on a 
measure of English proficiency such as TOEFL and/or on tests of 
developed verbal reasoning ability in English! as will be seen 
later, they do) . 



^According to information provided by the GRE program, a significant 
number of candidates, both citizen and foreign, during the 1978^79 
testing year may have confused codes used to indicate a GRE Advanced Test 
field with codes used to designate intended department or field of study* 
Since certain planned analyses of GRE score data involved classification 
of candidates by field, it was decided not to include data for the 
1978-79 testing year in the analysis. ; 
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^ TaMe 1 



Slse of Samples of Foreign (EFL and ESL) and Citizen-ESL Candidates 
Taking GMAT and GRE Aptitude Tests, Respectively , 
and Subsamples Taking an Admissions Test and TOEFL 1 



Test/ Candidates taking Candidates taking admissions 

Citizenship/ admissions test test and TOEFL 



primary 
language 


N 


percent* 


N 


« percent 

Percent* _ _ . . 

Overlap** 


CRE-fereigu ESL 


111172 


44,0 


2 ? 451 


62.7 


21,9 


GRE-foreign EfL 


U*ll 4 


43-8 


l p 369 


35-0 


12,3 


GRE-Clcizen ESL 


3 P 093 


12.2 


90i 


2-3 


2.9 


(Total GRE) 


25.379 


(100,0) 


3,910 


(100.0) 


(15,4) 


GMAT- foreign ISL 




37*8 


4,245 


75,6 


16,0 


GMAT-foreign EFL 


25,Q96# 


35.7 


1,312 


23,4 


5,2 


GHAT-Citi^en ISL 


18,69fi# 


26.6 


57@ 


1-0 


0,3 


(Total CHAT) 


70, 368# 


(100.1) 


5,614 


(100.0) 


(8,0) 


Note: GRE and TOEFL/ ORE data are 


for candidates 


taking 


the GRE 


Aptitude 



Test during 1977-78| candidates say have taken TOEFL during the 
period 1977-79- GHAT data are for candidates taking GMAT during 
1977-79| GHAT/TOEFL Candidates may have taken TOEFL at any time 
during the same two—year period, - * ^ T ■ 

* Base for percentage is number of candidates, by admissions test category.', 

** Base for percentages is number of candidates , by admissions test category, 
taking both an admissions test and TOEFL. For example p 2,451 GRE-foreign 
ESL candidates also took TOEFL, representing 21*9 percent of all GRE-foreign 
ESL candidates | probably an underestimate of ultimate ; overlap--©, g. t candi- 
dates taking TOEFL prior to 9/77 could not be identified since only a two= 
year file \t*a s used for the study. / = 

' J ' 

(I s Net included in all analyses. \ 1 

v 7 

i A 12.5 percent random sample of these candidates ik/ employed in the 
analysis. 
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Due to the small size of the citizen ESL subgroups in the TOEFL/GRE 
and TQEFL/GMAT samples , data for these subgroups were not included in 
all analyses* 

Data; general . For all candidates, scores were available on either 
the ORE Aptitude Test or the GMAT; scores on TOEFL were available for 
subgroups within the GRE and GMAT foreign samples* The test variables 
involved are as outlined below, 

GMAT verbal (GMAT-V), scaled 0-60 

GMAT quantitative (GMAT-Q) , scaled 0-60 

GMAT total (GMAT Total) f sealed 200 -800 

GRE verbal (GRE-V) scaled 200 - 800 

GRE quantitative (GRE-Q) , scaled 200 - 800 

GRE analytical (GRE-A), scaled 200 - 800 

TOEFL Listening Comprehension, scaled 20 — 80 
TOEFL Structure and Written Expression, scaled 20 - 80 
TOEFL Reading Comprehension and Vocabulary, scaled 20 - 80 
TOEFL Total, scaled 200 - 800 

o Only the ORE analytical ability measure is relatively new, 
Introduced in 1977, this measure is designed to tap student 
abilities to recognize logical relationships , to draw conclusions 
from a complex series of statements s to make inferences from 
'statements expressing relationships among abstract entities such 
as. nonverbal or nonnumerical symbols, etc** This measure eorre— 
lates above *7 with both GRE-V and GRE-Q (Conrad, Trismen, 
and Miller-, £977)* GRE-A has been found to have validity for 
predicting first-year grades in graduate school comparable to that 
of GRE-V and GRE-(|, in samples that excluded foreign students 
(Wilson, in press). 

A variety of nontest data (e.g* , answers to a variety of background 
questions) were available in the admissions-testcandidate files* ; The 
nontest data actually employed in the present analysis will be considered 
separately for GMAT and GRE samples* The GMAT and TOEFL/ GMAT analyses 
are described in Section 2; a generally parallel presentation for the ORE 
and TOEFL/GRE analyses follows in Section 3* Major trends in findings 
and general conclusions are considered in Section 4* 



^Beginning with the 1981-82 testing year^ a revised version of the 
analytical ability measure was introduced* 



Section 2* GMAT Foreign and TOEFL/GHAT Candidates: 
Selected Characteristics and Score Profiles 

According to data available on the GMAT general candidate population, 
the typical GHAT examinee is almost 28 years of age (mean - 27,9 years) 
and a substantias majority is male (about 74 percent). Less than half of 
the general candidate population during the three^year period 1975^76 
through 1977—78 (4 6,9 percent) who answered selected GMAT background 
questions indicated that they planned to study full time, and about one in 
six reported no M full time employment M when they last took the GMAT 
(Schrader, 1979), 

Table 2 shows comparable data for foreign ESL and , foreign EPL 
candidates (and also citizen ESL candidates) in the GMAT and TOEFL /GMAT 
samples* In evaluating the data in this table and others, it is important 
t o keep-in-mi nd ( a) th a t th m GM AT — s am pies i n elude individu a is in the 
TOEFL/ GMAT samples, and (b) that the GMAT analysis is based on a 12,5 
percent sample of foreign (EPL and ESL) and citizen ESL candidates* 

o The GMAT foreign and citizen ESL groitps averaged slightly over 27 
years of age; the TOEFL/ GMAT subgroups were younger, by a year or 
more on the average, than the average member of the general 
candidate population* 

o About eight in ten foreign EPL and foreign ESL candidates were 
male, as compared to about 74 percent of all GMAT candidates! only 
about two-thirds (67,9 percent) of the GMAT citizen ESL candidates 
were males, 

b As compared - to the general GMAT candidate population, proportion- 
ately more GMAT foreign candidates reported plans to study full 
time; among citizen ESL candidates, however, only about 44 
. percent planned full-time study, 

© Only about one— tenth (10,7 percent) of citizen ESL candidates 
reported no fuli^time work experience as compared to 22 percent of 
foreign EPL, 26 percent of foreign ESL, and 16,7 percent of GMAT 
candidates generally. 



GMAT Performance 

Measures of central tendency and variability (means and standard 
deviations) for distributions of GMAT verbal, quantitative, and total 
scores for foreign ESL, foreign EPL, and citizen: ESL candidates are shown 
In Table 3* Several features of these data are noteworthy, 

o On GMAT quantitative, the mean scores of foreign candidates do 
not differ from the mean for the general candidate population; 
EPL and ESL foreign candidates have identical means on GMAT-Q . 
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Table. 2 

Selected Characteristics of Foreign EFL* Foreign ESL, and 
Citizen-ESL Candidates In the GHAT and TOEFL / CHAT (T/GMAT) 
Samples, 1977-79 



Group 



Age in 
years 
Mean 



Male 



Pull -else 
itudy plan 



Ho full-time 
work exp 
% 



General CHAT 
popular ion* 

GHAT foreign =15 L 

GHAT foreign-EFt 

GHAT eitlsen-KSL 



100,000* 

3,322** 
3,137** 
2,337** 



27,9 

27-4 
27 .2 
27,3 



74,3 

80.0 
79.1 

67,9 



46.9 

' 89,0 
64,9 
44,3 



16.7 

26.0 
22.0 
10.7 



T/ GHAT forelgn-KSL 4,245 
T/GMAT foreign-EFL 1,312 
T/GHAT citiEen-ISL 57 



26.6 
25. 3 
26-7 



81,9 
84,6 
61,4 



89.0 
88,7 

75,0 



26,0 
29.6 
28.1 



*General GHAT population data are from internal program sources* Age 
data are for candidates tested during 197 5-76| other data are for 
candidates answering the background questions involved iurlng three 
testing years, 1975=76 through 1977*78, 

**Ns are for a 12,5 percent random sample of GHAT foreign (EFL and ESL) and 
citizen . ESL candidates with test scores, 1977-78 and 1978-79 combined. 

Note- General GMAT population data include multiple responses 
or records for repeating candidates. 
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Table 3 

Means and Standard Deviations of GHAT Scorei for All 
Foreign and for Citizes-ESL Candidates, and for 
Gemparahle Croups Taking loth GHAT and TOEFL i 
1977-78 and 197%-79 



Cand idate 
group 



GHAT-Q 



GHAT-Total 



Hean SB Mean SD 



All candidates* 
1977-1980 * 

GHAT foreign-ESL t 
1977-X979 

GHAT foreign-EFL s " 
1977-1979 

GHAT cltizen-ISL # 
1977-1979 

T/GHAT forelgn-ESLi 
1977-1979 

T/GHAT foreign-EPL, 
1977-197? 

T/GHAT citizen -ESL P 
1977-1979 



563,849 26 
** 



3 f 322 
3,137* 
2,337* 
4 t 24S 
1,312 
' . 57 



15, f 8,0 

24.2 9,8 

27.3 8,5 
If ,7 7,8 
20,2 9,1 
17,6 8.5 



27 . S 

27,0 f,3 

27,0 9,2 

26,0 8,0 

28,9 9*2 

27,5 9.9 

23-4 8.9 



Mean 



IB 



462 105 

380,4 100,6 

446.6 116,8 
466,0 100.4 

389.7 98,6 
417.9 114,8 

374.8 107,0 



*80-8l GHAT Guide to tW'Ute of GHAT Scores ftfata for all candidates) 
**Ns represent a 12,3 percent random sample of all foreign GHAT candidates 
and Eit-citiEen GHAT candidates for 1977-78 and 1978-79, Ms for 
T/GHAT samples indicate the total number ©f candidates identified as 
having taken both GMAT a nd TOEFL during the same period. 

Note i "All candidate" statistics include multiple records of 
repeaters, j ' 4 
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o Among foreign candidates, those reporting English as the best 
language have markedly higher mean GMAT verbal scores than their 
ESL counterparts, but their verbal mean is lower (by about 0.2 
standard deviations on the average) than that for the normative 
candidate population* 

o The mean for GMAT citizen ESL candidates on GMAT verbal is 
actually slightly higher than the mean for all GMAT candidates; 
their quantitative and total score means are about the same as 
those of the general GMAT population* 

o The pattern of GMAT performance of TOEFL/ GMAT foreign candidates 
is similar to that for the total GMAT foreign population (i.e,, 
EFL candidates higher than ESL on verbal) i but the verbal scores 
for the TOEFL/ GMAT foreign EPL group are lower on the average 
than those for foreign EFL candidates generally* 

o The very small group of citizen ESL candidates in the TO EFL /GMAT 
sample have lower GMAT^Q but somewhat higher GMAT*-V means than do 
foreign ESL candidates* However, they are quite different from 
the GMAT citizen ESL population, having substantially lower 
verbal, quantitative, and total scores* 

o On balance, it is clear that differences between all foreign ESL 
and all foreign EPL candidates on GMAT total are due solely to 
differences in their performance on the verbal section of the 
GMAT"dif f erenees that favor candidates who report that they are 
most fluent in English, "; \ 

Further perspective for assessing the level of performance of the 
foreign and citizen ESL candidates is provided below in the form of 
percentile ranks in the general candidate population (Educational Testing 
Service, 1980a) corresponding to the respective GMAT mean scores for the 
various groups. 



Citizenship/ 
language group 



Percentile ranks of mean GMAT scores for 



candidates taking 
GMAT 



candidates who 
also took TOEFL 





V 


Q 


Total 


V 


Q 


Total 


ESL 


14 


50 


21 


13 


59 


23 


EPL 


38 


50 


41 


25 


52 


31 


ESL 


51 


45 


48 


17 


33 


20 



From the foregoing it is evident that U.S. citizen candidates, even 
those who indicate they are less fluent in English than in some other 
ianguage s perform better on the verbal section of GMAT than self-reported 
foreign candidates, even. those who report they are "most" fluent in 
English. It can also be inferred that citizen ESL candidates have verbal 
scores comparable to th"8e of thfiir citizen EPL counterparts but that 



foreign candidates who report that they are less fluent in English than in 
some other language have markedly lower scores on the verbal teat than 
their counterparts who indicate that they are most fluent in English, 

It is also evident that the small group of citizen ESL candidates * 
taking TOEFL in addition to GMAT is not representative of the larger 
GMAT citizen ESL population. However, insofar as performance on GhAT is 
concerned, the TOEFL/GMAT foreign EPL- and ESL-candidate subgroups appear 
to be comparable to the general GHAT foreign EPL and ESL populations. 

GHAT Performance in Relation to Performance on TOEFL 

Over 5,000 GMAT candidates were identified by cross-file matching as 
also having TOEFL test records during the study period* The TOEFL/ GMAT 
sample includes both EPL candidates and ESL candidates- EPL candidates 
in the TOEFL/GMAT subgroup have a lower GMAT verbal-mean than that for all 
GMAT foreign EPL candidates, but a higher verbal mean than their ESL 
counterpart Si 

In this section, consideration is given (a) to the characteristics 
of the distribution of GMAT and TOEFL scores for foreign EPL and ESL 
candidates and (b) to the relationships between TOEFL and GMAT scores 
in these subgroups, with special emphasis on the relationship between 
TOEFL total and GMAT verbal. 

Characteristics of score distributions . Table 4 shows the means ang 
standard deviations of scores on TOIFL and GMAT, for designated groups* 
Also provided are percentile (centile) ranks of the respective means 
showing the standing of the typical TOEFL/GMAT foreign candidate (a) on 
TOEFL, relative to the score distribution for all prospective graduate- 
level applicants taking TOEFL between September 1978 and August 1980, &tid 
(b) on GMAT, relative to the score distribution for all candidates taking 
GMAT between October 1977 and July 1980. Several features of the data in 
Table 4 are noteworthy, j • 



f As noted at the outset, TOEFL f s candidate population is made up almost 
exlusively of individuals whose reported "native language" is not 
English. However, it is apparent that for graduate-^school bound candi- 
dates averaging somewhat over 25 years of age, the current "language of 
greatest fluency" may well be different from the historically relevant 
"native language" (e.g., that of the parental family and/or the larger 
society during an individuals early years). 

In the analyses involving TOEFL/GMAT candidates , data for candidates 
over 45 years of age and/or for whom the date of the undergraduate degree 
and the date of most recent testing differed by more than 15 years were , 
excluded. This was done to avoid the possibility of unusual outlier 
effects in samples of foreign candidates. Certain analyses, replicated 
in the unrestricted samples, suggest that this precaution was not 
necessary. 
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Table 4 

Means and Standard Deviations of Scores on TOEFL and GKAT 
and Gentile Ranks of Mean Scores for TOEFL/GHAT-foreigii 
Candidates 



EFL & ESL 



Gentile rank 
of mean** 



Teat 


<N*1197) 


(K^3918) 


(N-5il5) 


EFL 


ESL 


EFL a 
iSL 


Mean 


S,P* 


Hean 


S,D* 


Mean 


S,Tj* 


CHAT Verbal 


20,2 


9*0 


15,7 


7,7 


16,8 


8*4 


25 


13 


18 


GHAT Quantitative 


27,5 


9,8 


29,0 


9*2 


28.6 


1*4 


52 


60 


60 


GHAT Total 


418.0 


113.3 


389,8 


97.5 


396,4 


103,4 


31 


23 


26 


TOEFL Total 


589*4 


61*3 


541*8 


71.7 


552.9 


66.4 


89 


71 


77 


TOEFL Listening 


58*2 


6*9 


54.8 


6.9 


55,6 


6,9 


83 


71 


75 


TOEFL Writing 


58,7 


7,0 


53*0 


7*2 


54,3 


7*1 


88 


68 


72 


TOEFL Reading 


59, f 


6,4 


54,8 


7*1 


56,0 


6.8 


88 


67 


74 



Notei In analyses Involving the TOEFL/GHAT sample t candidates over 46 years 
of age and/or for whom the^ date of testing differed by more than 15 
years from the date of the undergf aduate degree were excluded. Thus, 
Ns In this table and others Involving this sample are less than those 
reported In earlier tables* 



*The percentile ranks for GHAT scores are relative to norms for all GHAT 
Candidates tested between 10/77 and 7/60| those for TOEFL scores are for 
all graduate-level TOEFL candidates tested between 9/78 and 8/80, For 
GHAT, normative sample means are 26 f 27, and 462. for V, Q and Total, 
respectively; standard deviations are 9* 8, and 105, For TOEFL, graduate- 
level means are 51.1, 49.4 , 51,0, and 505 for Listening Comprehension, 
Structure -and Written Expression, Reading Comprehension * and Total, 
rcspec t ivelyj and standard deviations are 6.8, 7.6 P 7*5 and 65* 
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o The typical GMAT/ TOEFL candidate was well above average In per- 
fonnance on TOEFL; the TOEFL total mean, of 553 is at the 77th 
centile in^the distribution of scores for TOEFL graduate-level 
candidates* 

a As expected s the typical EPL candidate performed better on TOEFL 
(with TOEFL total at the 89th centile) than the typical ESL 
candidate (at the 71st centile) . 

o EPL and ESL groups are separated somewhat more sharply by their 
TOEFL scores than by their GMAT verbal scores— —e.g. s in analyses 
not reported in detail , the point biserial correlation coefficient 
(computed as a measure of the degree of separation of the distri^ 
but ions of scores for EPL and ESL groups, respectively) for TOEFL 
total vs. EPL/ESL (1,0) was found to be .30, while the comparable 
coefficient for GMAT verbal vs. the EPL/ESL criterion was .23. 



o The typical TOEFL/GMAT foreign candidate (though average or better 
on GMAT-Q ) is well below the general candidate average on GMAT 
verbal — i.e. , the GMAT-V mean of 16.8 is at approximately the 
1 8th centile among all GMAT candidates* EPL candidates have 
somewhat higher average verbal scores (25th centile) than do ESL 
candidates ( 13th centile), 

Measures of central tendency and variability alone are not suffi^ 
cieritly descriptive of the distributions of scores of TOEFL/GMAT foreign 
candidates on the verbal measures under consideration here, as may be 
seen in Figures 1 and 2, These figures show the relative frequency of 
scores (i.e., the percentage of candidates) in various intervals of 
the scales for GMAT verbal and quantitative (Figure 1) and TOEFL reading 
comprehension and total (Figure 2) for foreign ESL and foreign EPL 
candidates* In Figure 1, relative frequency polygons representing the 
distribution of scores of ail GMAT candidates on GMAT— V and GMAT-Q are 
superimposed on the distributions for ESL candidates; in Figure 2, a 
similar comparison is shown for the TOEFL total distributions of all 
graduate-level TOEFL candidates and of TOEFL/GMAT foreign ESL candidates. 

o In Figure 1 it is evident that the distribution of GMAT verbal 
scores for ESL candidates is centered near the low end of the 
verbal score scale and tends to be pom 1 1 i v e 1 y s kewed — -i.e. , 
characterized by heavy concentrations of scores in two or three of 

. " . : / 

5 

The mean of 553 on TOEFL for these TOEFL/GMAT candidates is almost 
. identical to the mean (556) for some 9,000 TOEFL candidates tested during 
the study period who indicated that they were taking TOEFL because they 
planned to enter a graduate school of management (see Wilson, 1982a). 

^The distribution for all GMAT candidates is based on candidates tested 
during the 1975—76 testing year (available from Unpublished material 
provided' by the GMAT program) ; the TOEFL graduate— level distribution is 
derived from tables published in the TOEFL Test and Score Manual (1981). 
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the lowest i nervals with a gradual decline in frequencies over 
several high- %tr intervals . The GMA1-V score ..distribution for 
foreign EPL d candidates is less inclined toward positive skewness , 
even though ~ r^cro than threa^ f our chs of the distribution is below 
the alheand l&-date median* 

Generally gpeaK^ Ing^ these properties are eharacteri of dls£ribu^ 

tlonfi Of scoros on aany standardized test that is of well above average 
difficulty (or a giurven group, The distributions of TOEFL reading and 
total scof^afor thee se candidates^ however, exhibit different properties* 

o f i jy[o 2 a h*> owe that the distributions for both ESL and EPL 
candidates terr tig to be concentrated heavf ' y a bove the center of the 
seo rHist rifiT oution and that they exhit.w negative skewness^ri »e , t 
very high Q€r iicencracion in one or two higher intervals with a 
gr^clyU apre* ad of scores across several lower score intervals^^a 
tendency th« %. is particularly pronounced in the distributions for 
foreign jyPL ce, candidates * - 

T n e s e proper tie ^ s are characteristic of score distributions on any 
standsir/digeii test chtria t is of less than average "difficulty for a given 
group; and TOEFL cl&^ariy is of less than average difficulty for those 
TOEFL candidates w he^ also take GHAT * The fact that the scores of GHAT/ 
TOEFL candidates ar&e concentrated heavily in the lower range of the GHAT 
verbal scale and in the upper range of the TOEFL scale is significant 
because such zircum^m tances generally are conducive to diminished effl^ 
eiency of fcwu.ur ei#mt within the particular high- and/or low-scoring 
groups under eossife ..deration. Because GHAT/TOEFL candidates perform 
exceptionally well - on TOEFL and very poorly on GHAT verbal, neither 
t^st Is asKioaliy ^sellable for this particular population* Thus, their 
relatdon&Vilps ar% to some extent attenuated by pure "measurement** 
consider atMa as ^sil as by distortions attributable to the less than 
Ji norma^iva" feel of ^proficiency in English. 

Scorel Interre d a ^tlonshlps for EPL and ESL candidates * Using data 
for essentially e sale TOIFL/^HAT a aipl e as that considered here* 
Powers ( 198!)) eKamiooad the correlation between TOEFL and @LAT scores in 
analy&as tfciet did ©eot take into account differences in reported English 
language a talus --tisane correlations reported by Powers are shown in Table 
5, Table 6 snows r«n.ults obtained in the present study when correlations 
between the two sees of test variables were computed separately for EPL 



The extent of cenec^oritration of the scores of GHAT foreign candidates at 
the lowest range of tapche verbal scale is indicated by the fact that, among 
candidate© tested Muring 1975-76, almost one-half (48,5 percent) of all 
candidates with. GMAAT verbal scores of 12 or below were noncitizens who* 
during tfrat'testltags year s accounted for only 15*6 percent of the total 
candidate populatleon, and that foreign candidates with scores of 12 or 
below repreKAfced 1 surname 28 percent of all foreign candidates tested during 
that ym&z* 



and ESL candi.da!;fes sss groups that differ inaverage lave 1 of developed 
proficiency in English, as reflected in TOEFL, and in awetage level of 
developed verbal reasoning ability, as reflected in GHAT ver^bal* 



o Table 3 shows chat, in the undifferentiated sample, „ Che strongest 

relationship (r « *71) obtains between GMAT verbal =nd TOEFL total 
(a composite of measures of English language listenSLng and reading 
comprehension and knowledge of essential points of ISlnglish grammar 
and usage); GMAI quantitative snd TOEFL variables are nor closely 
related (e.g. , for TOEFL total vs* QiAT-Q , r « *3S3>* Due to the 
fact that GMAT total includes a ! " nonverbal" comp<3>neiit , the GMAT 
total relationship with TOEFL total (r = .66) is lower than the 
TOEFL /GMAT verbal relationship. 



From Table o, it is evident that TOEFl/GHAT verbal ec=rrelations are 
consistently higher in the EPL sample than in either the combined EFL/ESL 
sample (Table 5) or the ESL sample. 

c One reasonable hypothesis regard : ng such resul t_ s is that the 
scores obtained on GMAT__ve_rbal are lore reliable indicators of 
differences in verbal ability (less "contaminateoT" By^d ITI eyerie es~ 
in English proficiency * per se) forGHAT candidat^ss in the range 
of "developed English prof iciencj" represented- by the TOEFL 
score distribution of EPL candidal! than for & ;MAT candidates 
exhibiting, on the average a lover level of de\^reloped English 
proficiency (i.e., ESL candidates )* It should be recalled in 
this connection that the distribution of verbal scores for ESL 
candidates had a proportionately heavier concentr»tion of scores 
near the low end of the scale thandld the distrELbution for EPL 
candidates* 

Although not shown in the tables, the reiationshir»s between GMAT 
verbal and GHAT quantitative scores and between TOEFL ^cotal and GMAT 
, quantitative scores are relevant* 

o In samples of GMAT eandTda tes "genfi£illy s the corr&lation between 
GMAT verbaj and GMAT quantitative i§ reported to fc»e in the *55 - 
.57 range, w In the EPL saapie, the verbal^quantit^mtlve correla- 
tion was .60 and in the ESL" saffipleit was *50; the combined 
sample, the verbal^quanti titive correlation was ^essentially the 
same as that reported for the general candidate p=»opulation; the i 
correlation of TOEFL total with GMAT-Q ( approximately .4 in EPL, j 
ESL, and combined samples) was sosevhat lower tlman the verbal^! 
quantitative correlations* The fact that verba^Z^quantitative 
cor relations in these foreign sample! were like th&ose in general 
GMAT candidate sepias indicates that, despite the very depressed 
levels of GMAT verbal scores of foreign, candid at efc » their verbal 
scores still have suffieien/t psychoiGtrie viabiiifcy to exhibit a 



Unpublished internal data. 




Table 5 

Simple Correlation Between GHAT and TOEFL Scorei ^ 



TOEFL/GHAT-foreign Sample (EFL & ESL) 



_TOEFL score 



Structure leading 

CHAT score and CemprehefisiCfn 

Listening Written ttid 
j^mj^rebengion Expreasion ^i?€^ulBry fatal 

GMAT- Verbal ,58 ,66 *6? .71 

GHAT- Quan 1 1 1 a r_ i ve .29 . 37 , 39 ,39 

CHAT- Total .52 ,61 .64 ,66 



Note: Combined EFL and kSL eorrelati'.snS as reported by Powers (1980); 
K * 5,781 , 



Table 6 

Simple Correlation between GHAT and TOEFL Scores i 
EFL and ESL Subgroups s TOEFL /GHAT 



TOEFL score 

Structure Beading 

GMAT score and Comprehension 

Listening Written and 



Comprehension 


Expression 


vocsbularv 


Total 


CHAT=Verbal (EFL) 


.63 


*70 


,73 


,76 


(ESL) 


,53 


-61 


.66 


,58 


GHAT-Quan tit stive (EFL) 


,37 


.35 


,39 


,41 


CESL) 


*27 


.42 


' ,43 


' .12 


GHAT-Totsl CEPL) 


.58 


.61 


.65 


,63 


(ESL) 


,47 


.60 


• 63 


M 



Hotel Hs for subgroups are 1197 and 3918 , EPL and ESL, regp^ctively* 
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"no mnal" relationship with scores earned on the GMAT quantitative 
section, * 

TOEFL total and GMAT verbals A detailed analysis . Graphic 
portrayals of the TOEFL total/GMAT verbal relationship in EPL and ESL 
subgroups an is provided in Figures 3 and 4* Figure 3 is a cross- 
tabulation of scores on the two measures in the EFL sample; Figure 4 is a 
comparable cross- tabulation for the ESL sample* In each figure, a line 
has been drawn that connects the median GMAT-V scores for candidates in 
the various TOEFL score intervals* 

o The somewhat" closer association between TOEFL _and_GMAT-V. in the 
EFL than in the ESL sample is evident in the data. It is also 
clesr (a) that the relationship between the two sets of measures 
is not linear throughout the entire range of scores and (b) tliat 
tendencies toward nonlinear! ty are somewhat more pronounced in 
the ESL* sample (in which 38 percent of candidates scored below 
13 on GMAT verbal) than in the EFL sample (in which only 22 
per cent of the sample was concentrated in the lowest GMAT~V 
scorre intervals). 

A syst ^tnatie assessment of the relationships portrayed In these 
scattarplocs is provided in Table 7, wriich shows the mean level of the 
observed GHAT verbal scores for candidates in selected TOEFL score 
intervals. Also shown in the table are two sets of predicted GMAT means" 
estliated using TOEFL total as the predictor variable, One set of 
estimates m^m based on a regression equation derived under the assumption 
of a linear relationship throughout the range of scores; the other is 
based on an equation that takes into account the observed curvilinear! ty, 
which provides a "quadratic fit" for the data* In both Instances, 

equations were developed using data for the combined EPL and ESL samples* 
The actual GMAT verbal means were determined directly from the data shown 
in Figures 3 and 4. i 



9 For detailed analyses of the nature of the relationship; between each 
of the TOEFL section scores and GMAT scores in the total GHAT/TOEFL 
saaple, s^e Powers (1980). ■ j 

I ■ ' 

10 The computer program employed used TOEFL total intervals defined as 
follows: e.g., 200*0001 - 225.0000, 225.0001 - 275.0000. In printing 
out, the decimals were dropped* Thus* one score point should be added 
to the lower limit of each interval to indicate the] actual range 
included _ For example, 425-475 includes scores of 426-475, Inclusive* 

11 In the curvilinear equation, the actual TOEFL total score- and the square 
of the scrore are entered as predictors. This procedure "exaggerates" 
the contribution of scores in the higher range relative to scores in the 
loir range* The curvilinear estimates are as reported by Powers (19^0, 
Table 4} ^ based on quadratic equations using data for all TOEFL/GHAT 
foreign candidates, EPL and ESL combined. ; 
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Table 7 

Level of Observed GMAT-Verbal Scores In He lac ion to Level 
Expected from TOEFL Total Scare: IPL and ESL Subgroups 



TOEFL Mid- Estimated Observed GMAT-V mean for candi- 

interval point GMAT-V* dates in TOEFL interval 







Linear 


Curvi— 
linear 










ESL 


Combined 


626-675 


650 


25,6 


26,5 


28 


,1 


(416)** 


27 


.0 


( 334) 


27,6 ( 750) 


576-615 


600 


21,1 


20,8 


19 


. 7 


(409) 


20, 


, 7 


( 926) 


20,4 (1335) 


526-575 


550 


16,5 


14,9 


14 


, 7 


(186) 


15. 


,2 


(1190) 


15.1 (1376)) 


476-525 _ 


500 


12, 1 


10,8 


10 


.3 


(104) . 


11, 


.2 


( 900) 


..-.11*1_(1004)_. 


426-475 


450 


7,£ 


8,5 


7, 


,6 


( 61) 


8, 


.8 


( 387) 


8,6 ( 448) 


376-425 


400 


3,0 


n, a, 


5, 


. 8 


( 19) 


9i 


.0 


< 153) 


8,6 ( 172) 



*Linear estimates are based on a regression equation baaed on data for the combined 
sample* for predicting OMAT-V from TOEFL Total: Predicted V = ,0102 (Total) 33,0 
Curvilinear estimates are as reported by Powers (1980) s using a total-sample 
equation which included Total-squared to yield the best quadratic fit, 

**Number in parentheses is the number of candidates in the respective TOEFL Total 
intervals * 
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o Generally speaking, estimates based on the quadratic equation 
are closer to the observed means in both the EPL and the ESL 
groups than are the estimates assuming linearity* 

o Trends in the fit between estimated and observed means are quite 
similar for both EPL and ESL groups when estimates are based on 
TOEFL scores above the 426-475 interval; in the lower intervals, 
the linear estimates fit the observed GMAT-V means of EPL candi- 
dates more closely than those of ESL candidates* 

The general similarity of trends in the two samples suggests that, 
for purposes of predicting GHAT verbal from TOEFL total* separate treat- 
ment of EPL and ESL subgroups probably is not necessary, 

o If the mean TOEFL total scores for EPL and ESL groups (as shown 
in Table 4) are substituted in the total sample regression 
equation (reported in the note to Table 7), it will be found that 
the estimated and observed GHAT verbal means are essentially 
identical* Consistent with this, in other analyses not shown 
.in, the table, when English^language vs * non-English language 
dominance was us ed as a nominal predicto r vari able (EP L ^ 1 , ESL m 
0) in combination with TOEFL total to predict GMAT verbal, the 
resulting multiple correlation coefficient did not differ from 
that for TOEFL total alone* 

Summary of GHAT and TOEFL/GMAT Findings 

The GMAT and TOEFL/GHAT analyses that have been reviewed indicate 
the following* 

> Insofar as performance on GMAT quantitative is concerned, GHAT 
foreign and TOEFL/GMAT candidates cannot be distinguished from the 
general GMAT candidate population, Moreover, on GMAT-Q, groups 
differing in average level of English language prof iciency— i-e, , 
ESL foreign, EPL foreign, and "GNAT candidates generally 1 * (the 
last representing the range of English-language proficiency 
characteristic of management-school bound, English-speaking U*S* 
citizens)— perform equally well, on the average, and their overall 
(MAT-Q score distributions are quite comparable, 

o However, with respect to GMAT verbal, for all foreign candidates 
taking GMAT and within the subgroup also taking TOEFL, the average 
performance of both ESL and EPL candidates is lower than that of 
the GMAT candidate population generally; the average performance 
of ESL candidates , reportedly less fluent in English than in 
another language, is markedly lower | however, as TOEFL scores 
approach 600, the mean GHAT verbal score of TOEFL/GMAT candidates 
approaches that of the GMAT general population, 

o Differences in the average GMAT-V scores for EPL and ESL candi- 
dates in the TOEFL/GMAT sample are consistent with differences in 
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their average performance on TOEFL total; and scores on this 
measure are substantially correlated with GMAT verbal (r =.. .71 in 
the total sample, r =- .76 in the EPL sample, r = .68 in the ESL 
sample) « 

o The higher TOEFL/GMAT correlations in the EPL sample may be due 
to the possibility that, because of their greater average level of 
English proficiency, the EPL candidates' performance on GMAT was 
more reliably assessed than that of their ESL counterparts, who 
had lower average English language facility, 



o Generally speaking, interpretation of observed relationships 
between two tests, one of" which is of well above average diffi^ 
eulty ( GMAT— V ) and the other of below average difficulty (TOEFL) 
for the sample under consideration , is complicated by purely 
psychometric considerations unrelated to the "skills" the tests 
are designed to measure- GMAT candidates who also take TOEFL are 
typically at or above the 75th centile among graduate^level TOEFL 
candidates (EPL candidates typically scored near the 90th centile) 
but below the 25th centile on GMAT verbal. Theoretically, for 
... hig hly ... s elected groups of TOEFL candidates (like those taking 

GMAT) , the correlation between scores on a somewhat more difficult 
version of TOEFL and a somewhat less difficult version of GMAT 
verbal should provide better estimates of the level of relation* - 
ship between the skills being measured than those reported here. 

. On balance s the findings that have been reviewed suggest strongly 
that the lower level of English language proficiency is a major factor 
contributing to the depressed score levels of foreign - csndidatresv 
especially ESL candidates, on GMAT verbal relative to the level for 
GMAT candidates generally. Despite the depressed levels of verbal scores 
for foreign candidates, however, it is evident that these scores have 
significant psychometric viability— e,g* * they correlate substantially 
wi th TOEFL scores s and verbal^quantitative correlations are consistent 
with those observed in the general GMAT candidate population- 



Section 3, GRE Foreign and TOEFL/GRE Candidates: 
Selected Characteristics and Score Profiles 

The analyses in this section are based on foreign candidates taking 
the GRE Aptitude Test during the 1977^78 testing year and a subgroup of 
these candidates who also took TOEFL during the period from September 1977 
through August 1979* Table 8 provides information regarding selected 
demographic and academic characteristics of the samples under considera- 
tion* Data from the general GRE candidate population for the 1978^|| 
testing year (Wild, 1980) are provided for purposes of comparison, 
All data are from the GRE files* Since the number of TOEFL/GRE citizen 
ESL candidates was 1 imi t^d "~ a a"t a for crtizeh ESL- candidates were-not- 
included in the TOEFL/GRE analyses* 

o Judging from the data in Table 8, the GRE foreign candidate 
population as compared to the general GRE population was slightly 
older on the average and characterised by higher percentages 
of males* doctorate-degree seekers, candidates with some previous 
graduate study, and candidates planning to major in physical or 
biological sciences, In the TOEFL/GRE subgroups, varv high 
proportions of males were oriented toward physical and bioscience 
departments* As in the general candidate population, females were 
underrepresented among foreign candidates planning study in the 
biological and physical sciences and overrepresented among those 
oriented to humanities and social sciences. 



GRE Performance 

Measures of central tendency and variability (means and standard 
deviations) of scores on the _GRB Aptitude Test for GRE foreign EFL and 
ESL candidates, GRE citizen ESL candidates, and EFL and ESL subgroups 
taking TOEFL are shown in Table 9. Also shown are data for the GRE 
general candidate population* In evaluating these data it is important 
to keep in mind that the several groups differ, as is shown in Table 8, 
with respect to sex composition and relative concentration in graduate 
fields of study that differ in verbal and quantitative rap basis* 

o Foreign ESL candidates (who indicated that they communicated 
better in a language other than English ) generally, and that 
subgroup of GRE foreign ESL candidates who also took TOEFL, 
clearly had depressed GRE verbal means (more than 1*0 standard 
- deviation below the general population mean). . 



12 Strlctly comparable data were not available for the general candidate 
population tested during 1977^78. For the purpose of comparison with 
foreign -and citizen ESL groups, the 1978-79 "all candidates" data are 
deemed to be appropriate, ambiguities regarding coding for intended 

, major field notwithstanding (see note, page 3). V - 
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Selected Charaeteristics of GR1 and TOEFL/CRE Candidates (Foreign IPL» 
Foreign ESL, and Citi^en-ESL) i Data for If 77-78 Except as Noted 
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Means and Standard Deviations of GRE Aptitude Test Scores 
for All GRl^foreign sad GRE-dtizen ESL Candidates, and 
for Subgroups Taking TOEFL 



GRE ~ GRE- GRE- 

Group Verbal Quantitative Analytical 

Mean S,D, Mean 5*D, ~~ Hbm S.fc 



GRE * general* 


831,650 


479 


129 


518 


135 


496 


120 


Foreign EPL 


11*114 


435 


141 


543. 


144 


438 


139 


Foreign ESL 


11,172 


342 


95 


174 


145 


383 


114 


Citizen ESL 


3,093 


430 


127 


460 


140 


422 


133 


TOEFL/GR! 
















Foreign-IFL 


1.369 


386 


123 


603 


133 


406 


119 


Forel^ EPL 


2 f 45l 


345 


95 


606 


136 


400 


114 


Cottbined 


3,820 


360 


108 


605 


135. 


402 


116 



*From 1980-81 Guide to the Use of Graduate Record Examinations (ETS)— examinees 
tested between 10/77 and i/80, ^-^aaiaecs 
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□ Foreign EPL candidates (who indicated that they communicated 
better in English than in any other language) generally were 
somewhat below average on the verbal and the analytical measures. 

o Those GRE foreign EPL candidates who also took TOEFL (about 
one-siKth of all foreign EPL candidates) had substantially lower 
means on the verbal measure than did foreign EPL candidates 
generally* At the same time, TGEFL/GRE EPL candidates performed 
somewhat better on the verbal measure than did their TOEFL/ GRE 
ESL counterpartsp (Selection into the TOEFL candidate pool 
for the foreign EPL GRE candidate is associated with lower 
peTrformance^on - the verbal measure.) _ - 

o Foreign candidates, without regard to intended field of study, 
had higher quantitative means than the general candidate popula- 
tion, but substantially lower means oh the analytical ability 
measure; it is noteworthy that the foreign ESL candidates 
generally* and both ESL and EPL candidates in the TOEFL/GRE 
subgroup, performed exceptionally well on the quantitative test* 

Additional perspective for assessing the level of performance of the 
foreign and citizen ESL candidates is provided below in the form of 
percentile ranks in the general candidate population (Educational Testing 
Service, 19S0b) corresponding to the respective means, 

Percentile ranks of mean GRE scores for 
Citizenship/ candidates taking candidates who 
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Foreign EPL 
Foreign ESL 
Citizen ESL 





GRE 




also 


took 


TGEFL 
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Q 


A 


V 


Q 


A 


31 


56 


31 


23 


72^ 




15 


64 


14 


16 . 


72 


H 


36 


33 


20 


n.a. 




n.a*. 



From the focegoing it is evident that foreign ESL candidates 
generally, despite ^exhibiting a high level of developed ability to perform 
tasks of the type represented by items in the quantitative test, do not 
perform well on either the verbal test or the analytical test (whose items 
make a relatively heavy demand on English-language verbal skills)* 

Performance by intended field of study . As noted above, foreign 
candidates—especially foreign ESL candidates—have higher GkE-Q means 
than do GRE candidates generally. However, they more frequently intend to 
study in engineering , mathematics, and physical science fields (see Table 
8} , and candidates in these fields tend to have higher than average 
quantitative scores . Figure 5 shows profiles of means on the verbal, 
quantitative, and analytical measures for GRE foreign arid TOEFL/ GRE 
foreign EPL and ESL candidates classified according to ten byggad 

v * j *- ' - 
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intended-field categories*"" In assessing the differences in profiles 
for various groups, it is important to keep in mind that TOEFL/GRE candi^ 
dates constitute a subgroup of GRE foreign candidates* 

The contrast in the level of performance of foreign candidates on 
the quantitative measure as opposed to the verbal and analytical measures 
is strikingly demonstrated in the"- figure* 

o On the quantitative test, the profiles of means by intended area 
of study for foreign EPL and ESL candidates conform very closely 
in level and pattern to the profile for all GRE candidates; no 
noticeable quant i tative^score mean differences are associated 
with reported differences in reported English— language primacy 
when control is intr .duced . for intended field, 

o On both the verbal and the analytical tests, the pattern of means 
of foreign candidates by intended area is generally similar to 
the pattern of field means in the general candidate population 4 
but the profiles differ rather markedly in level , 

o On both the verbal and the analytical measures * foreign EPL 
candidates are seen to exhibit generally higher performance 
across all fields than foreign ESL candidates generally, 

Re l ative — to — Z^4^—£r&h=erm*Tt t=rrpi rr/si — fore ign EPL cantildates why 

also took TOEFL performed somewhat better on the verbal than on 
the analytical test. Note that the verbal profile for TOEFL /GRE 
EPL candidates is clearly separated in level from the profiles of 
GRE ESL and TOEFL/GRE ESL candidates * whereas the analytical 
profiles are not clearly distinguishable* 

Citizen ESL candidates, whose profiles £*:m not shown (see Appendix A 
for means by field) * had higher verbal and analytical means in all 
intended fields than did foreign ESL candidates and lower quantitative 
means for all iields than did either foreign EPL or foreign ESL 
candidates* 

On balance, the GRE performance data that have been reviewed indicate 
that foreign candidates who report eemmmicating better in English than 
in another language (EPL candidates) perform better on both the verbal 
and the analytical measures than do foreign candidates who report the 
communicate better in some other language (ESL candidates). The data also 
show that average performance on GRE quantitative does not vary across 



Appendix A shows the fields included in each category shown In the 
figure* Also included in Appendix A are tabular summaries for each of 
the groups under consideration (i.e., GRE foreign EPL* foreign ESL, and 
citizen ESL candidates* and TOEFL/GRE foreign EPL and ESL candidates)* 
showing for each intended area of study and for each teat (a) the number 
of candidates and (b) the mean and standard deviation of scores. 
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groups differing, by inference, in average level of developed English 
language proficiency whan control is introduced for intended field of 
study * 

GRE Performance in Relation to Performance on TOEFL 

Almost 4,000 GRE candidates also took TOEFL during the study period; 
the sample includes both EPL and ESL candidates- Foreign EPL candidates 
taking the GRE/who are selected into TOEFL candidacy (because they need to 
demons t rate / their proficiency in English as well as their performance oh 
GRE tests) have been seen to have substantially lower verbal means but 
considerably higher quantitative means than foreign GRE EPL candidates 
who do not take TOEFL , but somewhat higher verbal means than their ESL 
counterparts in the TOEFL /GRE sample, - . 

In this section, consideration is given to (a) the characteristics of 
the distributions of TOEFL and GRE Aptitude scores for TOEFL/GRE foreign 
ESL and EPL candidates and (b) the relationships between TOEFL and GRE 
Aptitude test scores in these subgroups* 

Characteristics of score distributions . Table 10 shows the means 
and standard davia tions^of scores" of TOEFL/GRE EPL and ESL candidates on 
- - the three-TOEFL -sections ( listening camp r ehension , s true t ur e and wr i t ten 
expression, and reading comprehension and vocabulary) and TOEFL total , arid 
centile ranks of the means relative to the population of graduate^level 
candidates who took TOEFL between October 1977 and August 1980; GRE data 
are also provided in the table (percentiles indicate relative standing in 
the general candidate population for the period October 1977—June 1980). 

o The typical GRE/TOEFL exaninee was well above average in perform^ 
ance on TOEFL; the TOEFL total mean of 559 for the combined 
ESL-EPL sample is at approximately the 80th centile for TOEF^. 
candidates planning to enter graduate^level degree programs* 

o As expected , the typical EPL candidate performed better on TOEFL 
(mean at 84th centile) than did the typical ESL candidate (mean at 
76th centile) • ■ 

o EPL and ESL subgroups are separated to about the same extent by 
TOEFL total as by ORE verbal; the point biserlal coefficient 
(computed as a measure of the degree of separation of EPL and 



The 559 TOEFL total mean for these TOEFL/GRE candidates is somewhat 
higher than the mean (534) for some 36,000 TOEFL candidates tested 
during 1977^1979 who indicated that they were taking TOEFL because 
they planned to enter graduate department as degree^seeking students. 
Means by broad area of intended department ware 536, 533, 526 * and 536 
for humanities, social sciences , biosciences and physical sciences, 
respectively (see Wilson, 1982a) • 
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Table 10 

Hsfis and Standard Deviations oT TOEFL and GRE Scores and 
Gentile Hanks of Mian lepras for TOlFL/GRl Candidates 



Foreign IFL Foreign E5L Combined Centile ranks 

K H » 2442 k -1808- e f seang* 



Mean S*IU Mean S,D, Mean S.D, EFL ISL Combined 



GRE-V 


386 


' 123 


345 


95 


360 


IDS 


23 


is 


1ft 


GRE-Q 


603 


133 


606 


136 


605 


135 


72 


72 


72 


ORE- A 


4 OS 


119 


400 


114 


402 


116 


24 


23 


23 


TOEFL 




















\ Total 


573 


64 


152 


61 


559 


63 


84 


76 


80 


Listening 


5S 


7 


55 


7 


55 


7 


76 


72 


72 


Waiting 


57 


1 


54 


7 


55 


7 


83 


72 


76 


Read ing 

j \ 


59 


1 


56 


7 


57 


7 


84 


74 


78 



*Cen§ile ranks of OKI means are relative to norms for general candidate 
population for the period 10/77 - i/80; for TOEFL means cent lies are 
relative to graduate-level candidates tested between 9/78 and 8/80. 
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ESL subgroups) for TOEFL total vs * EPL/ESL (coded 1,0) is .16, 
while that for ORE verbal vs. EPL/ESL is .18* However, they are 
essentially undifferentiated by performance on ORE quantitative 
and GEE analytical. 

Figures 6 and 7 point up characteristics of the distributions of 
TOEFL/GRE candidates 1 scores on the three GRE Aptitude measures and 
TOEFL total. - 

o Figure 6 shows that the distribution of GRE verbal scores for the 
TOEFL/GRE sample (EPL and ESL combined) is concentrated near the 
low end of the GRE scale and is positively skewed— i.e., within 
the verbal distributions the proportion of relatively low scores 
is much higher than the proportion of relatively high scores; this 
is true to only a slightly lesser extent for the analytical 
distribution* On the other hand f the quantitative distribution is 
centered above the general candidate median and tends to be 
negatively skewed* 

o The TOEFL total distributions for EPL and ESL subgroups in Figure 
7 show characteristics opposite to those of the GRE verbal and 
analytical score distributions in Figure 6. Reflecting the 
fact that on TOEFL total the means for EPL and ESL candidates, 

— ^respectively , are at- t^e- 76 th- and-80t^-cent-Jrles-rela:tive- -to- all 
gradua te^level candidates * these distributions are ^concentrated 
largely above the mid— range of the TOEFL scale and tend to be 
negatively skewed . 

Within a group for which given tests are either substantially above 
or below average in difficulty $ efficiency of measurement (i.e., the 
reliability of observed differences between group members) tends to be 
attenuated* Accordingly, neither TOEFL nor GRE verbal nor GRE analytical 
is maximally "efficient** in TOEFL/GRE foreign samples, a factor that 
should be taken into account in assessing the level and nature of rela- 
tionships between TOEFL and GRE Aptitude scores in the sample of foreign 
candidates taking both tests (see Figures 1 and 2 and related discussion 
of similar findings in the TOEFL/GMAT sample) . - 

TOEFL/GRE score interrelationships . Tables 11 and 12 summarize 
results of general analyses of the relationship of TOEFL scores to GRE 
Aptitude scores. Table 11 shows simple correlations of the respective 
TOEFL section and total scores with GRE variables; Table 12 shows results 
of multiple regression analyses design to assess (a) the relative 

-jHMTtribllff rhm FPepepMue TOEFT. mmrrian firnrgR to prediction of the 

GRE variables and (b) differences in relationships under assumptions of 
linearity and curvilinearity . Results are shown separately for the EPL* 
ESL* and combined EPL/ESL samples. 

o In Table 11 it may be seen that relationships between TOEFL 
scores and GRE verbal are systematically higher than TOEFL/GRE 
analytical relationships! TOEFL correlations with each of thes^ 
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Table 11 

fimple Correlation between Scores em TOEFL and.Seerii on the 
GRE Aptitude Test for SPLj and Combined Samples 





GRfi- 


-Vernal 


EFL 


uantitative 
ESL Comb 


GRS- 


-Analyt leal 


1PL 


XSL 


Comb 


EFL 


ESL 


Comb 


Tots! 


,74 


,66 


.70 


,21 


,21 


,21 


,64 


,61 


.62 


Listening 


,60 


,41 


.52 


.14 


,08 


,10 


,53 


.47 


,49 


Writing 


,69 


,56 


*S3 


,20 


,22 


.21 


.5^ 


.52 


,53 


Reading 


,72 


,67 


*70 


.23 


,25 


.24 


.61 


,58 


.59 



Table 12 



Relative Contribution of TOEFL Part Scores to Prediction of GRE Aptitude Scores 
in famplea Differing in Reported English-language Communication Skill 



Sample/Dependant 
Variable ' 


beta 
Listening 


t weights 
Writing 


a 

Read big 


Multiple 
Correlation 
(R) 


TOS'L 
Total 
(r) 


Quadratic 

m 


TOEFL 
Reading 
(r) 


Foreign ESL 
















H - 2.442 
















CRE-V 


10 


13 


52 


687 


658 


700 


672 


GRE-Q 


-14- 


09 


19 


259 


215 


242 


255 


GRE-A 


16 


15 


38 


611 


608 


622 


564 


Foreign MpL 
















H * 1,365 
















GRE-V 


17 


25 


40 


750 ■ 


745 


788 


717 


GRE*Q 


-04 


©s 


22 


233 


212 


216 


230 


GRE-A 


11 


IS 


34 


640 


639 


658 


609 



Foreign ESL + EPL 
H - 3,808 



GRI-V 


11 


20 


48 


718 


701 


749 ( 


695 


GRE-Q 


*09 


Of 


22 


251 


209 


228 


238 


CRE-A _ 


. 18 


14 - 


- 37 . 


~ - -617 


- 616 


630 


586 



Wetef Data are for foreign candidate* taking TOEFL during 1977-79 Who took GRi during 1977-78. 

•These are standard partial regression weights* Decimals have been ©aitted for the weights and 
correlation coefficients is the table, 
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variables are systematically higher in the EPL than in the ESL 
sample. 



© In the combined sample, coefficients for TOEFL total and reading 
comprehension vs. GRE verbal are equal (r ■ -70); in the ESL 
sample the coefficient for reading is slightly higher (,67 as 
compared to ,66) , and in the EPL sample the reverse is true 
(.74 and .72 for total and reading, respectively, vs. verbal), 

o TOEFL scores are only weakly associated with GRE quantitative 
scores, and the levtii of the relationship is essentially constant 
across EPL and ESL groups. 

Table 12 shows, for each sample, (a) the multiple correlation 
resulting from treating the three TOEFL section scores as a battery of 
predictors (rather than using TOEFL total) and the relative weighting 
of the respective TOEFL scores in analyses assuming linearity of the 
relationship between Aptitude scores and TOEFL scores , and (b) results 
obtained under an assumption of curvilinearity (i.e. , when TOEFL total and 
TOEFL total squared were treated as predictors) • 

o In the combined samples (foreign ESL and EPL), for example, it 
may be seen that when GRE-V is the dependent variable, the 
multiple correlation of the_ three TOEFL section scores is .718 
(as compared to the s imp 1 e ~c o r r ela t fo n~of- .701 -between -XQ-EF.L. 
total and GRE verbal) ; the largest standard regression weight 
was for reading (.48 as compared to .20 for writing and .11 for 
listening). When TOEFL squared is introduced, the TOEFL totai/GRE 
verbal relationship (quadratic R) increases to .749, from R - .716 
under the assumption of linearity, indicating seme departure from 
linearity of the relationship* 

o The data in Table 12, generally, suggest that there is little 
to be gained from treating TOEFL section scores as independent 
variables, rather than using TOEFL total, for the purpose of 
estimating the relationship between TOEFL scores and GRE Aptitude 
scores; however, they do point up the dominant role of reading 
comprehension among the section scores. The results also indicate 
that some improvement in estimation of GRE performance from TOEFL 
performance should be obtained using equations that include the 
squared TOEFL score element. ■ , 

Data on the relationships among the respective GRE Aptitude measures 
in samples that differ in average level of proficiency in English are 
Hshown^ below. ~ - ~ ~ ^ = . __ ... ■ — — — — _ 



Sample 

National (ITS, 1981) 
TOEFL/ GRE EPL 
TOEFL/GRE ESL 
Combined 





Relationship 






V.A 


q,A 


.51 


,72 


,69 


,31 


,75 


,56 


.24 


,68 


.56 


,26 


.70 


.56 
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o The correlation of GRE verbal with ORE quantitative (VtQ) is 
substantially lower in the TOEFL/ GRE foreign sample than in 
a national saaplt of GRE candidates, as is the GRE— Q/ GRE^-A 
correlation (Q ,A) . The verbal/analytical (¥,A) correlation, 
however, is relatively stable across samples differing in English 
language proficiency * \ 

TOEFL total and GRE verbal a more detailed examination . Figure 8 
is a 'cross- tabular" presentationXpf the relationship between TOEFL total 
and GRE . verbal scores in the TOEFL^GRE foreign sample. In Figure 8, the 
dashed line connects points representing the mean GRE verbal score for 
candidates in various intervals on IpEFL; the solid line connects points 
representing the median scores for the respective TOEFL intervals, Note 
that the intervals containing the means, of the respective reference groups 
have been marked by and those containing the sample means on the two 

tests by "**•'* Several features of the data are noteworthy, 

o As shown in Figures 7 and 8, the (marginal) distribution of GRE 
verbal is positively skewed, with a marked concentration of scores 
below the sample mean (the "**" interval) , while that for TOEFL 
total tends to be negatively skewed (note concentration above the 
550-570 interval with a scattering of lower scores), 

o It may be determined from Figure 8 (a) that only about 14 percent 

jpf t he f or_ei^ _c^aj^dT.dates hav e GRE verbal scor es^ in o r ^bove the 

GRE reference group mean interval and (b) that 85 percent of ~ these 
candidates scored in or above the TOEFL reference group mean 
interval—indicating, of course, that for TOEFL/ GRE candidates 
TOEFL is of considerably less than average difficulty while 
GRE verbal is of considerably greater than average difficulty. 

The curvilinear nature of the relationship between the two tests is 
evident from the trend lines connecting the means and medians of GRE 
verbal scores for individuals in the various TOEFL intervals, Note that 
the trend line of GRE— V means is consistently above that for the medians* 
indicating that the positive skewness observed in the marginal distribu- 
tion tends to characterize GRE distributions in the respective TOEFL 
intervals, - 

o The distributions of GRE scores for candidates in the higher TOEFL 
intervals have considerable symmetry , whereas GRE distributions 
for individuals in the Zower intervals of TOEFL tend to "pile up" . 
in the lowest intervals of the GRE scale. These trends reflect 
the fact that the GRE verbal performance of foreign candidates 
with higher TOEFL scores (e.g. , 600 or above) is comparable to 
j-h at 0 £ QRE candidlfees geneTanyT^her raster foreign catididarfces— 
with iower^scores on TOEFL, level of GRE verbal performance is 
depressed and the range is arbitrarily restricted because the GRE 
verbal scale does not permit downward assessment of differences 
among very low^scoring candidates. 
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o By Inspection of the trend line (and the mean values of GRE verbal 
for candidates in selected TOEFL intervals), it is .evident that 
mean GRE verbal scores tend to increase across successive inter^ 
vale on TOEFL, except for TOEFL intervals in the 310-430 range ; 
the observed mean GRE verbal scores for successive intervals in 
this range are almost identical, approximately 240. 

.The lack of relationship between TOEFL scores and GRE verbal 
scores for individuals in the lower ranges of TOEFL may reasonably 
b^ attributed to the essential unreliability of GRE verbal scores 
f o\r foreign candidates with - such low TOEFL (and GRE verbal) 
scores- However, for TOEFL scores above 430, GRE verbal means tend 
to .increase across successive TOEFL intervals at an apparently 
accelerating rate, with the highest rate of increase observed 
for \tOEFL intervals above 570. 

- " \ \ 

If it is noted that GRE verbal scores of 242 are at approximately 
the third cftntile in the national GRE population (i,e. , the population 
reference group for purposes of score reporting), the fact that scares 
above this point begin to covary positively with TOEFL scores suggests 
that, despite the very depressed nature of the GRE verbal score distribu- 
tion for TOEPL/GRE foreign candidates, even very low GRE verbal scores 
appear to have potential utility as measures of differences in developed 
verbal reasoning ability, within the population of TOEFL/ GRE fore j^ti 
^a4i44r4^^e^ ^(^=n^ 1 ^by— in 
population) , 

© The pattern of findings reported here with respect to the level 
and nature of observed T0EFL/GRE verbal relationships in a 
sample of "regular" TOEFL /GRE candidates is generally ^similar to 
that reported by Angelis , Swinton, and Cowell (1979) for, a 
sample of foreign TOEFL candidates who volunteered to take the 
GRE verbal test, (See Appendix B*3 for cross^tabulations * ) 
However , there are some differences in findings. These appear 
Yto be a function of the fact that the sample of volunteers 
differed rather markedly from . the present sample with respect to 
level and range of scores on GRE verbal (mean/S*D# values of 
/ I 274/66 as compared to 360/107 in the present sample) , as well as 
on TOEFL (523/69 as opposed to 559/62)* For example, an estimated 
39 ' percent of the sample of 186 volunteers scored below 242 on 
GRE verbal* This is the mean GRE verbal score of candidates in 
thfe present T0EFL/GRE sample corresponding to TOEFL scores below 
431, -the empirically identified point at which GRE verbal scores 
begin to covary systematically with TOEFL total*\ In the volunteer 
sampre , syst ema tic^ovaria^ J.QjR_Jhe^ 
~~ - .. was not ; apparent in TOEFL ranges below approximately 470, as 
compared to about 430 in the present sample* These differences 
- ^ point up the critical importance of identifying the parameters 
(for TOEFL and GRE tests) that delineate the joint distributions 
of" "scores for foreign nationals who regularly are "selected 
Into*' joint membership In the pools of candidates taking TOEFL 
and GRE, 



Summary of GRE and TOEFL/ GRE Findings 



The GRE and TOEFL/GRE analyses that have been reviewed indicate the 
f ollowing % . 

p Foreign candidates who are selected into the GRE and TOEFL/GRE 
candidate pools perform as well or better on the GRE quantitative 
test than do GRE candidates generally; however, these same candi- 
dates earn quite low average scores on^ both GRE verbal "and GRE 
analyticaJL* Among foreign candidates s those who are reportedly 
less fluent in English than in another language (ESL candidates) 
have lower verbal and analytical scores than do those reporting 
English as the language of greatest fluency (EPL candidates). 

o The foregoing patterns hold when comparisons are controlled 
for intended field of graduate study. On the GRE quantitative 
test, profiles of means \ for foreign EPL, foreign ESL, and the 
all^candidate population, by field, are essentially identical with 
respect to both level and pattern. Profiles of GRE^V means for 
EPL and ESL groups tend to parallel the all-candidate profile but 
at depressed levels; EPL profiles are generally at a higher level 
than ESL profiles. 

o GRE foreign EPL candidates who also take TOEFL have lower verbal 

_ arifi afiaiyfirai TTj^anft, but substantially higher quantitative 

means, than do GRE foreign EPL candidates generally. TOEFL/GRE 
EPL candidates have a higher verbal mean than do their ESL 
counterparts, but the quantitative and analytical means of EPL and 
ESL subgroups within the TOEFL/GRE sample are- essentially the 
same* EPL and ESL subgroups are separated to about the same 
extent by performance on TOEFL as by performance on GRE verbal 
(point biserial coefficients of ,16 and ,18, respectively). 

o GRE citizen ESL candidates have GRE verbal scores roughly compar- 
able in level to those of GRE foreign EPL candidates generally , 
but their means on GRE quantitative and analytical are lower than 
those of both GRE foreign candidates and th^ general GRE candidate 
population, \ h 

o TOEFL total correlates substantially with fcRE verbal (r m .70 in 
the combined EPL and ESL sample) and with the analytical measure 
(r * .62); TOEFL/verbal correlations are higher in the EPL sample 
(r - ,74) than in the ESL sample (r = -66) , \as are TOEFL/analyt- 
ical correlations, but to a lesser extent t -61 vs. ,64 for ESL 
and EPL) ._ Mgher TOEFL/verbal correlation injthe EPL than in_chr 

... ESL sample may be attributed , hypo thetically , to more, reliable 
assessment of differences in GRE verbal (and lanalytical) for the 
EPL than for the ESL sample, for which GRE-*V is an extremely 
difficult test. 

! / - , ■■ 

o Inferences about the degree of relationship between the skills 
measured by TOEFL and those measured by GRE verbal (or analytical) 



are limited by the fact that the GRE ver t ' f~r%i analytical) 
measures are of much greater than average difficult" for the 
TOEFL/GRE sample, as well as by the fact that these candidates are 
also highly selected in terms of their scores on TQEFL^-^over 70 
percent of the sample scored above the TOEFL graduate— level 
reference—group mean and below the GRE all^candidate reference 
group mean* . ^ 

o Detailed analysis of the relationship between TOEFL total and GRE 
verbal scores in the TOEFL/GRE foreign sample indicates that GRE 
. verbal scores tend to increase with increases in TOEBL scores 
except in the range of TOEFL scores below approximate ly 430* For 
TOEFL/GRE "foreign candidates below that TOEFL score* mean scores 
on GRE verbal were essentially stable (at approximately 240) over 
successive, lower TOEFL intervals* 

On balance* the findings s trongly sugges t that limited English 
language facility, per se, is a factor (perhaps the major factor) cdntrib^ 
uting to the depressed More -4-eveis of foreign candidates on GRE verbal 
and GRE analytical tests* Analyses involving the GRE verbal test, 
however, suggest that, despite the depressed performance levels of foreign 
candidates, their GRE verbal scores may have significant psychometric 
viability: except in the lowest ranges of the TOEFL/GRE scales, verbal 
scores tend to increase systematically with scores on TOEFL* 
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Section 4, Overview and Evaluation of Findings 

This study was undertaken (a) to assess the performance of GHAT 
and CHE foreign candidates on the two admissions tests in relation to 
sel 1 s repo r ted primary iangage (English vs * other), and (b) to analyze 
relationships between periormance on those tests and performance on TOEFL 
for examinees who took both the GMAT or the ORE and TOEFL* 

Analyses that were essentially parallel in nature were conducted for 
each admi sslons - test sample. First, attention was focused on differences 
in the levels and patterns of admissions test scores obtained by EPL and 
ESL candidates— subgroups assumed to differ in average level of "developed 
proficiency in English*" Then, within each admissions^ test foreign 
candidate sample, detailed analyses were conducted in subsamples identi- 
fied as also having TOEFL records during the 1977-1979 study period. In 
these subsamples , analyses were conducted to assess the- extent and nature 
of the relationships between scores on the respective admissions tests and 
scores on TOEFL in EPL and ESL subgroups, Detailed consideration was 
given to the character t sc ic s of the distributions and joint distributions 
of scores on TOEFL and the cvu admissions tests* 

Ihe findings have been examined in detail in the preceding sections* 
General trends in findings were quite similar across the respective 
admissions- test samples* Major common themes and related conclusions or 
observations are summarized below* 

o Regardless of d i £ f e r encer - cul t ur al , educational, linguistic, 
and other—that may exist between foreign candidates and the 
"majority" candidate populations of the admissions tests, 
the average (mean) performance of foreign candidates on the 
quantitative sections of GRE or GMAT equals or exceeds that of the 
respective admissions-test populations. This holds not only for 
foreign EPL and ESL samples generally, but also for subgroups 
selected into TOEFL candidacy. 

o In sharp contrast, the means of foreign candidates, both EPL and 
EJjL, on the verbal sections of GRE and GMAT (and also on the 
analytical GRE measure) fall substantially below the means of the 
respective admissions-test populations and, by inference, well 
below the average level of verbal performance expected on the 
basis of their performance on the quantitative tests (comparable 
to that of the general candidate populations), however, among the 
foreign admissions-test candidates generally (and to a lesser 
ex teut in the subgroups also taking TOEFL) , the mean verbal 
admissions-test performance of EPL candidates exceeds that of ESL 
candidates* 

o Within both the GMAT and GRE foreign candidate populations, the 
subgroups also taking TOEFL are highly selected members of the 
TOEFL candidate population with respect to level of developed 
English language proficiency as measured by TOEFL performance 
(mean TOEFL total of 533 and 559 for GMAT and GRE samples, 
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respectively), EPL TOEFL candidates outperform their TOEFL ESL 
counterparts on TOEFL (as wall as on the verbal sections of the 
two admissions tests); percentile ranks (graduate-level distribu= 
tion) of TOEFL total means were 89 and 84 for EPL> and 71 arid 76 
for ESL candidates in the TOEFL/GHAT and TOEFL/GRE samples, 
respectively* 

o TOEFL total scores are substantially correlated with GMAT and GRE 
verbal scores (and to a slightly lesser extent with GRE analytical 
scores) s but are only weakly correlated with the quantitative 
sections of the admissions tests* Generally speaking, TOEFL/ 
quantitative^test correlations are weaker than verbal/quantitative 
admissions^ test correlations * 

o Obser/ed TOEFL/verbal admissions-test correlations were systemat- 
ically higher in EPL than in ESL subgroups. Overall correlational 
results were remarkably similar across admissions^test samples. 
In the GRE sample, TOEFL total/GRE verbal coefficients were 
,74, ,70, and *b& for EPL, combined EPL/ ESL, and ESL analyses, 
respectively; comparable TOEFL total/GMAT verbal analyses yielded 
coefficients of .76 ^ * 7 1 , and * 68 . Correlations computed under 
the assumption of curvilinearity were slightly higher than 
the foregoing* 

o The higher TOEFL/verbal correlations in EPL than in ESL subgroups 
may be attributed, hypotheticaliy , to the likelihood that, because 
of the greater average level of English proficiency in the EPL 
samples t the verbal admissions tests are psychomettficaliy more 
appropriate for these samples than for their ESL counterparts, 
for whom the verbal admissions tests are particularly difficult, 

© Inferences from the level of the observed correlations regarding 
the degree or nature of relationship between the skills measured 

Jb y TOE F L^_a n d_ _ t ho se measured b y the verbal admissions tests are 

limited by the fact that both GR^^^nd^LCT-TT are of-mueh~gre»fcer— 
than average difficulty for TOEFL/admissions^ tes t candidates, 
while TOEFL is of considerably less than average difficulty for 
these candidates* Reliability of observed scores is attenuated Jg 
groups scoring at the extremes on standardized tests generally. 



A5 In an analysis of TOEFL/GRE verbal relationships in a sample of TOEFL 
candidates who volunteered to take GRE verbal. Angel is, Swinton, and 
Coweli (1979) computed reliability coefficients for the verbal test and 
its components* The coefficient reported for GRE verbal was .78* as 
compared to a reported coefficient of .94 for a group of GRE candidates 
who were "native speakers" of English* the reliability of the reading 
comprehension subsection of GRE verbal in the TOEFL/GRE sample was 
reported as .47, compared to .84 reported for a general reference group. 
The GRE verbal measure was unusually difficult for this group of TOEFL 
candidates. 



o Interpretation of observed relationships between TOEFL and 
verbal admissions tests is complicated by basic psychometric 
considerations unrelated to the skills measured by the tests* 
Theoretically, easier "levels" of verbal admissions tests and 
"somewhat more difficult "levels" of TOEFL should provide more 
reliable assessments, hence better bases for evaluating observed 
relationships between TOEFL and standard verbal admissions tests. 

o Detailed analyses of the relationship between TOEFL total and the 
two verbal admissions tests suggest that , despite the "depressed 1 " 
and positively skewed nature of the verbal score distributions for 
TOEFL/admissions^tes t candidates , the verbal admissions=tes t 
scores tend to have significant psychometric viability in the 
sense that they tend to increase sys tematically with scores on 
TOEFL, except in the lowest sector of the joint TOEFL/verbal test 
score distribution. From this juncture of very low TOEFL and very 
low admissions scores; the reiationahip between TOEFL total and 
the verbal scores becomes increasingly robust in the upper TOEFL 
score range — -in which the mean verbal scores of candidates begin 
to equal or exceed the mean for the general admissions^ res t 
candidate populations* 



General Conclusions 

The pattern of findings warrants several general conclusions, which 
are offered tentatively in the form of working hypotheses for future 
consideration. 

1* With respect to the verbal sections of s tandard admissions tests, 
except for scores in the lowest centiles, the observed verbal 
scores of foreign candidates have a useful degree of "interpret- 
ability" in the sense of permitting valid inferences regarding 
withln-groxip differences in relative level of developed "verbal 
reasoning ability 8 *— i.e. , the verbal performance of one foreign 
candidate may meaningful ly be compared with the observed- verbal 
performance of another but not with that of majority (English- 
speaking) candidates* 

o This hypothesis is strengthened by evidence from scattered 
studies of the predictive validity of scores on verbal tests 
(such as GEE verbal, GMAT verbal, and ihe Law School Admission 
Test), Results of these studies indicate that, within samples 
of foreign students, differences in verbal scores are signifi- 
cantly related to differences in actual academic performance 
as measured by grade-point average (e*g* , Harvey & Lannholm, 
1961; Harvey & Pitcher, 1963; Sharon, 1971; Sehrader & 
Pitcher, 1976; Wilson, 1979; several unpublished GMAT validity 
studies) . 

o Additional support for this, hypothesis is provided, albeit 
indirectly, in findings of a study by Alderman (1981). In a 



sample composed primarily of colleger-bound high school E8L 
students in Puerto Rico, Alderman found that the relationship 
between College Board Scholastic Aptitude Test verbal (SAT— V) 
scores and verbal scores on the Prueba de Aptitud Academica 
(PAA~V)^^constructad specifically to parallel the verbal 
content of the S AT— V measure in the Spanish language — 
increased substantially as scores on TOEFL increased. In 
the present context it is most relevant to note (a) that 
Alderman's sample had a mean TOEFL total of about 435 (almost 
two standard deviations below the mean of the TQEFL/GRE and 
TOEFL /GM AT candidate populations) , (b) that the PAA-V/SAT-V 
relationship was strongest at TOEFL score levels in the range 
above 500 (well below the TOEFL means for the candidate 
populations under consideration in the present study) s and 
(c) that in the total Puerto Rican sample, which again was 
characterised by a comparatively low average TOEFL total 
score, the PAA=V/5AT~V relationship was still moderately 
strong , r ■ *67* It is reasonable to infer that, in samples 
of admissions test candidates averaging above 550 on TOEFL, 
obtained scores on verbal measures in English are likely to 
have a useful degree of "interpretability - " 

Although foreign ESL candidates earn much lower average scores on 
verbal admissions tests than do their U.S. citizen counterparts, 
it does not follow that differences in average performance on 
these tests validly reflect differences in functional level of 
developed English^ language verbal or analytical reasoning ability 
or that s after foreign ESL students are admitted to graduate 
programs, their average level of academic performance will be 
like that of U.S. citizen students with comparably low verbal (or 
analytical) scores. 

o A major factor associated with less than "native familiarity" 
with the English language is likely to be diminished speed 
of .performance of tasks involving verbal processing (e.g., 
lower reading speed)* Available evidence^ although limited, 
indicates that foreign candidates have lower completion rates 
than U*S. citizen candidates on both GRE verbal (Angelis. 
Swinton, & Cowell, 1979) and GMAT verbal (Sinnott, 1980)— 
i*e, p that these tests have a greater degree of speededness 
for foreign than for U.S* citizen candidates. 

o These findings are consistent with research on speed of 
processing and language dominance, reviewed by Dornle (1980), 
that focused primarily on M *.*decoding (comprehension) and 
encoding (production) of spoken language" (emphasis added)* 

o Research is needed to establish the average levels of perform— 
ance attainable by foreign ESL candidates on verbal admissions 
tests under essentially una peed ed conditions . Given the 
logical and theoretical significance of speed of verbal 
processing in a nondominant language, there would also appear 
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to be a potentially important role in English^language testing 
for separate measures of speed and level of comprehensions 



o It is reasonable to hypothesize that, under normal conditions 
of academic life, foreign ESL students, on the average, maybe 
able to compensate for their relatively low speed of verbal^ 
task completion by spending more time— on^task than their IKS , 
citizen classmates. If so, their general level of academic 
performance would tend to be higher than that expected baled 
on their performance on a "speeded" standardised test of 
verbal ability, 

o Since many quantitative test items are embedded in an Engliih 5 - 
language matrix, it is possible that these measures may alio 
be somewhat more "speeded" for foreign than for U.S. citizen 
candidates, Generally speaking , research is needed to 
ascertain the relative speededness of quantitative ability 
- measures for foreign and citizen candidates , notwithstanding 
the fact that the quantitative performance of foreign and 
citizen candidates is, on the average, at the same level. 

o In any event, questions regarding the comparative predictive 
value of standard admissions tests for foreign and UiS* 
citizen students require empirical answers, It is important 
to extend the comparatively limited body of validity study 
evidence currently available by conducting studies designed to 
assess the comparative performance of foreign and U.S. citizen 
students who have similar scores on verbal and other standard 
admissions tests. 

With respect to TOEFL, the findings of this study contribute 
further evidence of the test's value (a) for reliably asseasing 
differences in develor^^ proficiency in English for admissions^ 
test candidates whose F^fifl ah language background is questionable 
and (b) for the purpc ^ of identifying subgroups within such a 
candidate population whose verbal adml ssions^ tes t scores are 
likely to have differing degrees of interpretability and useful- 
ness in the admissions process* 

© Institutional— level studies of the relationship between scores 
on TOEFL and scores on standard admissions tests should be 
conducted since the characteristics of the joint distribution 
of TOEFL /admissions- test scores for foreign applicant popula- 
tions may vary considerably across institutions* For example, 
each institution should determine* for its own population of 
foreign applicants , the TOEFL score level above which scores 
on TOEFL and scores on verbal admissions tests begin to vary 
together* systematically* by developing scatterplots such as 
those shown in this study for TOEFL*-admissions test candidates 
generally. 




46- 



o From the point of view of English language testing per se s as 
auggested earlier, consideration should be given to the 
potential role of separate measures of speed and level of 
comprehension involving both written and spoken materials* 

Because foreign nationals for whom English is not the dominant 
language are an increasingly significant segment of the popula- 
tion of candidates taking standardized admissions tests such as 
the ORE and GMAT, it is important to reiterate the need for 
research on (a) test speededness for foreign candidates and (b) 
the predictive validity of the admissions tests for foreign 
students, with attention to the role of TOEFL scores in the 
interpretation of validity study findings. 
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Appendix A 



A ¥ l Groups of Disciplines Used in "Major Area M GRE Analysis 

A . 2 GRE Aptitude Test statistics, by sex and intended graduate field: 

TOEFL/ GRE EPL candidates, 1977-78 

A*3 GRE Aptitude Test statistics, by sex and intended graduate field: 

TOEFL/ GRE ESL candidates, 1977-78 

A, 4 GRE Aptitude Test statistics for citizen ESL GRE candidates 

1977=78* By sex and Intended graduate area of study 

A*3 GRE Aptitude Test , statistics for foreign EPL candidates, 1977-78. 

By sex and intended graduate area of study 

A, 5 GRE Aptitude Test statistics for foreign ESL candidates, 1977^78i 

By sex and intended graduate area of study 
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Croups ef maelpllnes Used is "Major area" 0KE Analysis 
(ef. Figure 5) 
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Appendix B 



B-i Scat terplots of GRE verbal and TOEFL total scores of foreign EPL 

and ESL subgroups, respectively 

B.2 Scatterpiot of GRE analytical and TOEFL total scores for combined 

EPL and ESL samples 

B,3 Scatterpiot of GRE verbal and TOEFL total scores in a sample of 

TOEFL candidates volunteering to take GRE verbal 
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